Online Reviews: Empirical Generalizations



Qianyun (Poppy) Zhang

*Joint work with Prof.Vishal

Consumer Reviews

  • Product reviews are considered as one of the most trusted sources by consumers

  • Online reviews are important for all stakeholders
    • Individual Products/Firms (e.g. United Airline, Authors of book)
    • Platform (e.g. Expedia, Amazon)
    • Fellow consumers (helps purchase decision)
    • Review writer (satisfaction of expression, potential revenue source)

Empirical Context

  • Online Reviews from several sources
    • Amazon : 142.8 million reviews, May 1996 - July 2014
    • Glassdoor: for all US firms (last scrape July 2016)
    • IMDB full data (1990 - 2016)
    • Yelp: few major cities (global)
    • Travel: Expedia, Homeaway

Approach

  • Broad approach: extract attributes at 3 broad levels
    • Review (content of the review)
    • Reviewer (attributes of the writer)
    • Product (attributes of the product)

Approach

Review Reviewer Product
Star Rating Reviewer Name Product Name
Length Gender Average Rating
Text Geography Price
Time Total Reviews (heavy/light) Broad Category
Helpfulness Popularity/Sales

helpfulness

Summary: number of reviews @ Amazon.com

Text Mining

  • Text features: standard NLP

  • Readability

  • Sentiment

Readability

  • SMOG(Simple Measure of Gobbledygook) years of education a person needs to understand a piece of writing
  • The Automated Readability Index: ratios of word difficulty and sentence difficulty
  • Coleman–Liau Index: weighted difference between average number of letters and average number of sentences
  • Flesch readability: average sentence length and average number of syllables per word
  • Cunning-Fog: average sentence length plus percentage of hard(long) words
  • Higher score, lower readability

Non-readability

Highly correlated in the data

Female Reviews are more Readable


Female
Male

Sentiment

  • Valence shifter
    • We follows the approach developed in Rinker, T. W. (2017)
    • Amplifier generally increase the polarity of the sentiment words and negator flip the sign; pause punctutation, such as semicolon, and adversative conjunctions, such as “but”, also shift the tone
    • Shifter accounts for around 20% words in Trump tweet and Austen’s books
    • Average sentiment and sd. sentiment; sentence level

sentiment example

Average Sentiment by Star Rating & Length of Review

Deviation in the sentiment

Research Question

  • What makes a review helpful?
  • Syntactic and Semantic Gender differences in Review Writing
  • Someone is Watching: Impact of Amazon’s Policy Change on Reviews

  • Today: What makes a review helpful?

Summary: mean helpfulness

What makes a review helpful?

Star Rating

  • Imagine the average rating for a product is 4.5, which review do you think is more helpful? 1 star, 3 star or 5 star?

  • “There is a general bias, based on both innate predispositions and experience, in animals and humans, to give greater weight to negative entities (e.g., events, objects, personal traits)” Rozin & Royzman (2011)

Star Rating and Helpfulness


Female
Male

Higher Star rating, higher helpfulness

Generalzation Method

  • Product has at least 5 reviews; review has at least 5 votes; before 2014

  • Relative star: absolute above 0.5

  • All other numeric variables are scaled to mean 0

  • Processing on HPC clusters at Stern

  • helpfulness=ReviewLevel+ReviewerLevel+ProductLevel

Longer review is more helpful

Mixed: Non-readability

Positive Reviews is more helpful

Deviation in sentiment is more helpful

First review is more helpful

Review for product with high variance in rating is more helpfulness

Review for products with a lot reviews is less helpful

Deviance from overall rating is less helpful

TODO

  • Generalization of review helpful to broad specturm of categories
  • Additional reviewer attributes
  • Additional text attributes
  • Controlling for product effects
  • Explore interactions

End.

Thank you

(let’s dance)

Literature

  • Star rating
    • Higher star rating leads to higher helpfulness
    • Deviation also leads to lower helpfulness (Chen, Dhanasobhon, and Smith (2008); Mudambi and Schuff (2010); Pan and Zhang (2011); Racherla and Friske (2012); Baek, Ahn, and Choi (2012); Filieri (2015))
    • Confirmation bias
  • Review Length
    • Longer review is more helpful (Mudambi and Schuff (2010); Pan and Zhang (2011); Racherla and Friske (2012); Baek, Ahn, and Choi (2012))
    • Longer reviews embed more and deeper information

Literature

Mixed findings

  • Readability
    • Grade or vocabulary level needed to understand certain text corpus
    • Some studies found readability has no or little marginal effect (Hu et al. (2012); Korfiatis, García-Bariocanal, and Sánchez-Alonso (2012))
    • Some studies found higher readability higher helpfulness (Anindya Ghose (2010))
  • Popularity of products
    • Higher popularity leads to lower helpfulness (Mudambi and Schuff (2010))
    • Higher popularity leads to higher helpfulness (Anindya Ghose (2010); Baek, Ahn, and Choi (2012))

Literature

  • Product Category
    • Reviews for search products are more helpful than experienced products (Baek, Ahn, and Choi (2012))
    • Product Category can moderate the effect of review length and review rating (Mudambi and Schuff (2010); Korfiatis, García-Bariocanal, and Sánchez-Alonso (2012))

Literature: Summary

  • Modelled only some of the factors, not all of them
  • Mostly used categories: DVD, cameras, MP3 palyers, softwares, cellphones, printers, movies
  • Mostly focused on one fraction in time
  • Constraints of data: Amazon has very good anti-scraping protection Very hard to maintain and analyze 150 million reviews

Readability

Publishing while Female: Gender Differences in Peer Review Scrutiny Erin Hengel 2017

Reference

Anindya Ghose, Panagiotis G. Ipeirotis. 2010. “Estimating the Helpfulness and Economic Impact of Product Reviews.” Ieee Transactions on Knowledge and Data Engineering, 1–15. doi:10.1109/TKDE.2010.188.

Baek, Hyunmi, JoongHo Ahn, and Youngseok Choi. 2012. “Helpfulness of Online Consumer Reviews: Readers’ Objectives and Review Cues.” International Journal of Electronic Commerce 17 (2). Taylor & Francis: 99–126.

Chen, Pei-Yu, Samita Dhanasobhon, and Michael D Smith. 2008. “All Reviews Are Not Created Equal: The Disaggregate Impact of Reviews and Reviewers at Amazon. Com.” Com (May 2008).

Filieri, Raffaele. 2015. “What Makes Online Reviews Helpful? A Diagnosticity-Adoption Framework to Explain Informational and Normative Influences in E-Wom.” Journal of Business Research 68 (6). Elsevier: 1261–70.

Hu, Nan, Indranil Bose, Noi Sian, and Ling Liu. 2012. “Manipulation of online reviews : An analysis of ratings , readability , and sentiments.” Decision Support Systems 52 (3). Elsevier B.V.: 674–84. doi:10.1016/j.dss.2011.11.002.

Korfiatis, Nikolaos, Elena García-Bariocanal, and Salvador Sánchez-Alonso. 2012. “Evaluating content quality and helpfulness of online product reviews: The interplay of review helpfulness vs. review content.” Electronic Commerce Research and Applications 11 (3). Elsevier B.V.: 205–17. doi:10.1016/j.elerap.2011.10.003.

Mudambi, Susan M, and David Schuff. 2010. “What Makes a Helpful Review? A Study of Customer Reviews on Amazon. Com.” MIS Quarterly 34 (1): 185–200.

Pan, Yue, and Jason Q Zhang. 2011. “Born Unequal: A Study of the Helpfulness of User-Generated Product Reviews.” Journal of Retailing 87 (4). Elsevier: 598–612.

Racherla, Pradeep, and Wesley Friske. 2012. “Perceived ‘Usefulness’ of Online Consumer Reviews: An Exploratory Investigation Across Three Services Categories.” Electronic Commerce Research and Applications 11 (6). Elsevier: 548–59.